1. Field of Invention
The invention relates to a method for presenting fisheye-camera images, and in particular to a method integrating the multicollimator metrology and cartography in order to systematically describe the projection mechanism of the fisheye camera and to transform the fisheye-camera images into ones suitable for advanced visual applications.
2. Related Art
The camera systems in the field of artificial vision preferred using lenses with a narrow field of view (FOV) in order to obtain images approaching an ideal perspective projection mechanism for precise measurement. The perspective projection is usually a basis to deduce the camera's parameters while modeling the barrel distortion with a polynomial in the variable of the image's dimensions. The obtained intrinsic and extrinsic parameters can be employed to the visual applications in the quest for better precision, such as 3-D cubical inference, stereoscopy, automatic optical inspection, etc. These applications, however, currently have a common limitation of narrow visual angles and insufficient depth of field.
A fisheye camera (also termed a fisheye image sensor, FIS) can capture a clear image with a FOV of over 180 degrees, but a severer barrel distortion comes along. Because the projection function of the FIS is far from the prospective projection, the optical parameters cannot be deduced by those methods, which directly relate to the rectilinear mechanism of the perspective projection, for the normal cameras. Technologies developed for the usual visual disciplines resulted in no capability to process the images of the FIS.
Eventually, the panospherical imaging skipped from using the FIS (also called a dioptric sensor) to alternatively developing various camera systems with complex reflective optical elements (also called catadioptric sensors) as compensation. These solutions employed optical components such as reflectors or prisms to take panoramic views; for instance, the technologies were disclosed in the U.S. Pat. Nos. 6,118,474 and 6,288,843 B1. However, the catadioptric systems often elongate the ray traces, complicate the image-forming mechanism and attenuate the imaging signals due to indirectly taking the reflective images through the added optical elements. A blind area will be unavoidable at the center of an image because of the front installation of the reflective element.
For expanding the FOV, the camera system with a mechanical pan-tilt-zoom motoring function is another solution in the related art, which takes surrounding images in a row to achieve a panoramic view, for instance, the technology disclosed in the U.S. Pat. No. 6,256,058 B1. Or, contrarily, a number of cameras are employed instead to simultaneously capture images in different directions for seaming a panorama. However, the first method of a rotation type cannot take a whole scene at the same time so that a drawback of asynchronism remains. Furthermore, the volume of both systems is hardly to be shrunk to approach a hidden function or to take a close-range view; not to mention the heavy weights of the camera bodies consume more electricity and the rotating device is easier to get out of order. In addition to the extra cost of multi-cameras, the sampling and integration of the images from individual cameras still present many further problems.
Thus, adopting the lenses with a very wide FOV to take a whole scene in a single shot is a tendency of this kind of camera systems while considering many practical requirements in applications. An image-based algorithm aiming at the FIS assumed that the lens conforms to a specific projection mechanism so as to deduce the optical parameters without any calibration target. With reference to FIG. 1A and FIG. 1B, wherein FIG. 1A expresses the imageable area 1 of a FIS in a framed oval/circular region and FIG. 1B is the corresponding spatial projecting geometry of FIG. 1A, both figures note the zenithal distances of α, which are the angles respectively defined by incident rays and the optical axis 21, and the azimuthal distances of β, which are the angles surrounding the optical axis 21 (or the principal point C while on the image plane) by referring to the prime meridian 13, or the mapping domain 13′ of the prime meridian 13 in FIG. 1B. The azimuthal distances of the incident rays laid on the same radius of the imageable area 1, or in space on the same azimuthal plane (like the sector determined by the arc C′D′F′ and two spherical radii) are postulated invariant, like points D, E, F corresponding to points D′, E′F′ in FIGS. 1A and 1B.
In addition to the specific projection mechanism, the image-based algorithm furthermade several basic postulates: first, the imageable area 1 is an oval or a circle and the intersection of the major axis 11 and minor axis 12 situates the principal point, which is cast by the optical axis 21 shown in FIG. 1B; second, the boundary of the image is projected by the light rays of α=π/2; third, α and ρ are linear related, wherein ρ, termed a principal distance, is the length between an imaged point (like the point A) and the principal point (the point C). For example, the value of α at the point A is π/4 since it is located at the middle of the image radius, and therefore, the sight ray of α=π/4 indicates its corresponding point A′ in FIG. 1B; it is the same with point C and C′, point D and D′, point E and E′, etc. An imaged point can be denoted as (u, v) in a Cartesian coordinate system or as (ρ, β) in a polar coordinate system while taking the principal point as the origin.
Although the mapping mechanism was not really put on discussion in the algorithm, it is actually the equidistant projection (EDP) whose function is ρ=kα where k is a constant and is the focal length constant if the FOV is exactly 180 degrees (totally simplified as EDP π hereinafter); in fact, the EDP π is the basic postulate in the image-based algorithm. The focal length constant can be accordingly obtained by dividing the radius of the imageable area 1 with π/2.
Based on those postulates, the planar coordinates (u, v) on the imageable area 1 can be easily related to the spatial angles (α, β) of the corresponding incident rays. In light of the EDP mapping properties, an imaged point can be treated as a datum axis to turn into a transformed image without any further parameters needed. The U.S. Pat. No. 5,185,667 accordingly developed a technology to present FIS images in a hemispherical field of view (180 degrees by 360 degrees). This technology has been applied in endoscopy, surveillance and remote control as disclosed in U.S. Pat. Nos. 5,313,306, 5,359,363 and 5,384,588. However, the focal length constant was not included in discussion in the above researches.
Major parts of the mentioned image-based postulates, however, are unrealistic because many essential factors or variations have not been taken into consideration. First, there are several possible natural projection functions of the FIS and various probable ranges of the FOV, shown in FIG. 2; wherein the often-seen postulated EDP π is just one special case so it is unreasonable to lock all the projection geometries on the EDP π. From the curves in FIG. 2, the differences are obviously increasing along the growing zenithal distances respectively between the EDP and the other two projection functions: the stereographic projection, formulized as ρ=2ƒ×tan(α/2), and the orthographic projection, formulized as ρ=ƒ×sin α. Second, the projection π is doubtful and difficult to evaluate since the shape of the imageable area 1 is always presented as a circle no matter what the angular scale of the FOV is. A third factor concerns the errors caused in locating the image border. The radial decay of the radiometric response is an unavoidable phenomenon in a lens, especially when dealing with a larger FOV. This property will induce a radial decay on the image intensity, especially occurring with some simple lenses, so that the real boundary is extremely hard to set in the bordering effect. Consequently, the simple image-based algorithm is possessed of not only modeling errors but also practical limitations while deducing the optical parameters, so that the situated principal point could be unstable and the modeling ends up with a low accuracy.
Moreover, Margaret M. Fleck Perspective Projection: The Wrong Image Model, 1994 demonstrated that the projection mechanisms of lenses hardly fit in with the ideal models in the whole angular range in practice; otherwise, optics engineers could develop lenses with special projection functions, such as the fovea lens, in light of the different requirements from applications. Thus, imposing the postulate of the EDP π on all FIS is extremely forced.
It is obvious that the related art didn't probe into the projection function of the FIS and didn't locate the border accurately. It resulted in a low spatial confidence while processing the image, and also kept the applications from advanced development. The present invention will carefully look into this subject and formulate a camera-parameterized procedure out of ideal image-based postulates so as to precisely obtain the optical parameters and to exactly transform FIS images on the basis of the obtained parameters. Thus, the advanced applications can also be achieved accordingly.